An Introduction to Algebraic Semiotics, with Application to User Interface Design

This paper introduces a new approach to user interface design and related areas, called algebraic semiotics. The approach is based on a notion of sign, which allows complex hierarchical structure and incorporates the insight (emphasized by Saussure) that signs come in systems, and should be studied at that level, rather than individually. A user interface can be considered as a representation of the underlying functionality to which it provides access, and thus user interface design can be considered a craft of constructing such representations, where both the interface and the underlying functionality are considered as (structured) sign systems. In this setting, representations appear as mappings, or morphisms, between sign systems, which should preserve as much structure as possible. This motivates developing a calculus having systematic ways to combine signs, sign systems, and representations. One important mode of composition is blending, introduced by Fauconnier and Turner; we relate this to certain concepts from the very abstract area of mathematics called category theory. Applications for algebraic semiotics include not only user interface design, but also cognitive linguistics, especially metaphor theory and cognitive poetics. The main contribution of this paper is the precision it can bring to such areas. Building on an insight from computer science, that discrete structures can be described by algebraic theories, sign systems are deened to be algebraic theories with extra structure, and semiotic morphisms are deened to be mappings of algebraic theories that (to some extent) preserve the extra structure. As an aid for practical design, we show that the quality of representations is closely related to the preservation properties of semiotic morphisms; these measures of quality also provide the orderings needed by our category theoretic formulation of blending.